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I. INTRODUCTION 



The systematic study of documented aircraft problems experienced 
by operational personnel in the field receives little allocation of the 
human factors research being done in the Navy today. This seems sur- 
prising when one pauses to reflect that military aerial operations are 
composed of complex man-machine systems functioning in extremely 
competitive situations. The environment which pits a competitor versus 
an opponent (whether an actual physical enemy or simply nature) is 
likely to contain vicissitudes as unsettling as imaginable. Yet most 
expect maximum mission performance from the system at all times and 
place demands upon it accordingly. A necessary condition for such 
optimum performance is that man, the system’s most variable and least 
predictable component, must be allowed to apply his diverse skills to 
the situation at hand without distraction. Yet, in the actual evolution 
of a mission, system elements supporting man can operate below opti- 
mum performance at any given time. It is this suboptimum perform- 
ance which can distract an aviator's attention from a critical task and 
lead to error and mission performance degradation. Usually these 
errors are attributed to the human. However this paper adopts an 
alternative approach, namely considering inadequate aircraft design 
as a major contributing factor to suboptimum performance. It is well 
within the realm of the Human Factors discipline to study these errors, 
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determine causes, develop procedures /methods for alleviation, and 
finally to communicate the results to the design engineer. 

Thus there is a need for a human factors method which can sys- 
tematically collect, classify, and analyze data specifically chosen to 
aid the designer in both identifying and solving man- machine interface 
problems. Meister and Sullivan (1967) concluded that current human 
factors information supplied in the design stage is entirely "too verbal, 
too general and non-applicable. " In fact, Meister and Farr (1966) 
reported in an earlier study of the perception designers have of such 
data: 

"Human Factors problems - which are not usually verbalized 
as such - are settled largely by the designer's placing him- 
self in the role of the equipment operator. Designers have 
great difficulty in anticipating operational problems that may 
result from design parameters and are unable to apply evalua- 
tive criteria to completed designs. " 

More recently Wherry (1975) pointed out that current techniques, 

". . . rely heavily on the expertise of the human engineer 
to provide 'good' estimates for the data required by the 
various techniques and methods ... if the information 
input into the technique is unreliable and invalid then the 
output results will also be unreliable and invalid. " 

Thus it is clear that a data pipeline is sought which could provide the 

design engineer with more objective information concerning operational 

problems than he could generate subjectively himself. 

A question that might be asked in a search for such data is, "What 

parameters significantly influence operator performance in the field? " 

The present thesis represents one approach to this question, using 
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responses elicited from the users for obtaining evaluative data on 
operational systems. It is felt that any form of evaluation in system 
development should include human considerations as well as engineering. 

In the area of Naval aircraft cockpit design this implies that Naval 
Aviators and Naval Flight Officers (NFO'S) should be canvassed for 
reports of design discrepancies. It is recognized that reports of design 
problems submitted by the fleet personnel have been a basis for limited 
modification in the past. Yet too often the human engineer has been 
faced with single reports with which he must decide if a discrepancy 
is an isolated problem or, in fact, truly fleet-wide. The collection and 
classification of a large number of fleet generated design discrepancies 
is a necessary step in providing a sound, documented basis for modify- 
ing existing equipment and designing new systems. 

The type of information volunteered by fleet operators is highly 
subjective in nature. This led to a search of existing techniques for a 
procedure to classify and analyze such data. This thesis will demon- 
strate that the Problem Incident Technique can successfully organize 
this new information into categories that can aid the Navy in four areas: 

1. To improve the quality of decisions in developing new Naval 
aircraft. 

2. To aid the design engineer during periods of retrofit of 
existing Naval aircraft. 

3. To provide better integration of hardware and personnel 
(the man-machine system) in Naval aircraft, 

4. To correct design deficiencies early in development programs 
of new aircraft. 
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II. BACKGROUND 



Since World War II military systems have undergone dramatic 
periods of retrofit and reorientation during their operational lives. 
These changes have occurred in response to such variables as changes 
in the threat, technology alterations involving new or upgraded equip- 
ment, or of the deployment of systems in new operational environments. 
There is little doubt that operators in the field would have benefited 
during these periods from a comprehensive analysis of their perceived 
problems . 

Such an analysis across various Naval aircraft types could also 
aid the design engineer faced with evolving a weapon systems develop- 
ment program for new aviation platforms. As an example consider the 
approach to such a program which has been proposed by Wherry (1975) 
and is presented in Figure 1. 

This program extends from the premise that the only defensible 
rationale behind the development of future airborne weapon systems 
is to counter enemy threats as perceived for some given time frame. 
Thus the first phase consists of defining the operational requirements 
needed to counter the expected threat parameters. The next phase, 
mission scenario requirements, discusses how the Navy might per- 
ceive the engagement with enemy forces unfolding with time. Third, 
system capability requirements are described based on the various 
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FIGURE 1 

AIR WEAPON SYSTEMS DEVELOPMENT PROGRAM 
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scenarios. Finally the subsystem requirements may be defined as 
"an aggregation of components to accomplish various functions which 
taken together provide portions of the needed system capabilities" 

(Wherry, 1975). This may also be viewed as the first of the design 
stages since it conceptualized how the system might be organized. It 
is at this level that the value of supplying fleet generated data surfaces. 
Here it can be blended with (see Figure 1. ) human capability and engineer- 
ing state-of-the-art inputs to provide a real world sieve through which 
design concepts can be screened. 

No claim is made that all man-machine problems can be caught by 
such an approach. But fleet responses can provide a foundation for 
identifying and diagnosing many otherwise ignored problems which 
affect mission success (as determined by the humans charged with 
eliciting performance). It is recognized that most critical and urgent 
problems do not exist unheeded for long in the normal weapon system 
design process. However, this technique can potentially identify and 
document them much earlier - prior to the prototype stage and thus 
prior to the need for an expensive Engineering Change Proposal. Also 
it could highlight at the outset factors which are ordinarily not document- 
ed in operational field reporting and which when later identified as 
troublesome downstream in the design process are not deemed worth 
the expense to change. 
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m. DESIGN DEFICIENCIES AND PILOT ERROR 



The practical goal, the payoff, in developing fleet generated data 
for identifying critical problem areas involves the reduction of system 
performance errors. Error may be defined as simply a mistake, a 
deviation from correct procedure, which results in lost time, damaged 
equipment, personal injury, or any form of mission degradation. 

The Human Factors attitude towards error tends toward the premise 
that you cannot divorce the man from the system designed for him. Man 
is a system component - and both the strongest and weakest element in 
the aviation system at that. He is the strongest because he is the most 
flexible and adaptable in a given situation. He is the weakest because 
his performance is unreliable; he cannot perform the same task again 
and again in an identical manner. Nonetheless, man must be relied 
upon for effective performance if the system is to accomplish its tasks. 
To this end the goal of Human Factors is to optimize the design of 
equipment so that the user's efficiency will be at its peak. 

Although this thesis employs a method which concentrates on errors 
in equipment design rather than human error itself, it is necessary to 
realize that a causal relationship exists between the two. Ricketson, 
Kennamore, and Callen (1975) provide a functional definition of pilot 
(human) error and its causal elements (see Figure 2). 
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FUNCTIONAL DEFINITION OF PILOT ERROR 
FIGURE 2 



Items one through eight identify basic environmental elements which 
influence the aviation system and are potentially error producing. Item 
two is highlighted in Figure 2 since it represents the focus of this paper, 
namely vehicle /equipment design. When any of these elements exceeds 
its normal tolerance level, the pilot must correct for this aberration 
while continuing to perform his present ongoing tasks. This can result 
in an overload condition for the aviator which, if it occurs at a critical 
moment, can induce error. The outcome will be various degrees of 
performance degradation. 

A similar viewpoint was expressed by Meister (1971). He deter- 
mined that three general causes of error could induce an overload 
condition. First, error could arise because of an operator’s psycho- 
logical or physiological state - fatigue, lack of capability, lack of 
motivation. These were called ’’operator-induced" errors. Second, 
lack of proper maintenance programs, supervisory practices, facilities, 
or any other deficiency in total system planning could result in "system- 
induced" errors. Finally a state conducive to human errors could be 
induced upon the aviator by design problems in individual equipment; 
these were termed "design-induced errors. 

It is the third category of error, henceforth called design discrep- 
ancies, which is highlighted and analyzed by the Problem Incident 
Technique. These are simply problems with equipment design charac- 
teristics which create special difficulty for the operator and which 
substantially increase the potential for error. 
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IV. HISTORICAL PERSPECTIVE 



The employment of techniques utilizing field type data to analyze 
system performance problems is not a new development in military 
aviation. Fitts and Jones (1947a) were early investigators of such 
methods and analyzed 460 incidents of pilot error involving the opera- 
tion of aircraft controls. Later Fitts and Jones, (1947b) collected and 
analyzed 270 pilot error experiences in reading and interpreting aircraft 
instruments. Their findings concluded that the design of aircraft equip- 
ment must take into account the capabilities and limitations of the human 
operator. 

Later applications of the technique involved a flight safety orientation. 
Korchin and Patterson (1949) collected 497 accounts of "close calls" 
which were reported by pilots for an evaluation of flight safety media. 
Vasilas et al (1953) defined these "close calls" as "near accidents" and 
devised a standard report form for collection of Air Force personnel 
data. Significantly, they concluded that a group orientation approach 
(defined in section V. b. ) was a highly effective method of data collection 
for such a study. Ricketson et al (1973) surveyed accident investigation 
reports for U. S. Army aviation over a fifteen year period. They devel- 
oped an experimental human error reporting form in the course of their 
study and concluded that the resultant data held promise for providing 
a clearer identification of mishap-causing system elements. 
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Rabideau and Ritchie (1968) employed open-ended questioning of 
Air Force personnel in interview situations for a study of human engineer- 
ing problems incurred during tactical air operations in Southeast Asia. 

A classification of problem commonalities and causative factors resulted 
in more than two dozen problem areas. Atkins (1969) attempted to collect 
data inputs for developing new and revised criteria of Air Crew Station 
Geometry for application in U.S.A. F. aircraft. The research team 
studied a broad range of sources including accident data, unsatisfactory 
reports, and aircraft inspections. They concluded that the most fruitful 
efforts were associated with aircrew generated data. 

It is noteworthy that each of the preceding reports cited anonymity 
as a crucial requirement for any data collection format. Shapiro et al 
(I960) modified this when applying subjective techniques in a missile 
system test context. They placed the interviewee in the posture of being 
an anonymous third person; thus he was allowed to report errors he had 
observed others commit. This depersonalization mollified the feelings 
of respondents that their "admission" of performance errors might 
somehow be used against them. 

In spite of their differences, all of the preceding studies illuminated 
mission event incidents in which the performance of any system or sub- 
system was such as to cause some form of mission degradation. The 
remaining sections of this paper will study a technique which utilizes 
design discrepancies for the illuminating incidents. Although the objec- 
tives of the above works are somewhat different from the present thesis, 
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nonetheless they provided approaches and procedures which proved of 
value in promoting the present analysis. 
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V. PROBLEM INCIDENT REPORTING TECHNIQUE 



Van Cott and Kincaid (1972) state that the essence of a human 
engineering test consists of answering such questions as the following: 

"1. Can the operator use the equipment? 

2. If so how well? 

3. If not what is considered the problem by the operators who 
are part of the system? " 

The basic premise put forth is that equipment can not be tested and 
evaluated independent of the human beings who will work with it. 

The Problem Incident Reporting Technique (Meister and Rabideau, 
1965) is aimed specifically at answering the third question. It is closely 
related to, but not precisely the same as, the Critical Incident Technique 
proposed by Flanagan (1962). 

The Critical Incident Technique as originally postulated consisted 
of a procedure for analyzing and classifying what are, in essence, 
anecdotally reported observations. For the most part Flanagan used 
these for developing job requirements; thus he classified examples of 
both effective and ineffective behavior. However, the main concern of 
this paper is to collect problem incidents (which might be construed as 
"negative" incidents as opposed to "positive" items) for the purpose of 
alleviating poor design performance. It is these problem incidents 
which illuminate operator difficulties and presage breakdowns in mission 
effectiveness; thus, they can be used to identify the types of design 
behavior which should be carefully evaluated by the human engineer. 



20 



Meister (1965) proposed that problem incidents, when used in 
conjunction with the Problem Incident Reporting Technique, could be 
of value in indicating the following: 

1. Events which under adverse conditions could jeopardize 
safety of personnel and/or damage equipment; 

2. Human errors in operating equipment; 

3. Operator difficulties in performing mission tasks; 

4. Inadequate system parameters (i. e. job environment, 
equipment design) that affect system performance. 

The remainder of this paper is an application of this technique to the 

area of Naval aircraft design. 

A. DATA SOURCE 

Daniels (1976) suggested that the U. S. Naval Aviation Safety 
School at the Naval Postgraduate School, Monterey, California, provided 
a convenient data source for questions of aviation design. Classes of 
twenty to forty Naval Aviators /Naval Flight Officers convene at six week 
intervals. Each section is composed of a professional and recently pro- 
ficient group of fleet aviators that mirrors the spectrum of Naval air- 
craft types and communities. It thus provides an inexpensive data 
source that suffices for the initial problem conceptualization. 

It is recognized that such data will be biased somewhat since 
safety school entrants in general tend to be more proficient than the 
fleet norm; they have perhaps mastered a design problem that could 
hinder the "nugget" aviator. Yet this paper is concerned with establishing 
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the feasibility and value of the Problem Incident Technique in order to 
justify its projection to the squadron level (as a fleet-wide source of 
data). To this end the safety school provides in a sense a "worst case" 
source of information. After all if the more experienced element of 
the fleet identifies discrepancies they certainly cannot be excused as 
solely pilot inexperience. Such errors must be regarded as relevant 
and worthy of further study. 

B. METHOD 

A method of exploration was sought which could uncover what things 
operators found troublesome with their equipment and could also provide 
a vehicle for further hypothesis formulation. Thus a method was needed 
to facilitate the collection and processing of this type of information. 

The basic steps involved are presented in Figure 3 and provide a guide 
for this entire study. Phase II, the analysis section, is closely aligned 
to the Critical Incident Technique (Flanagan, 1962), which was later 
refined by Meister and Rabideau (1965) into the Problem Incident Report- 
ing Technique. 

Because of the cost in time and personnel, alternatives to individual 
interviews were sought for data collection. As noted earlier Vasilas 
et al (195 3) suggested that a group orientation method was highly effec- 
tive in facilitating the collection of reported incidents. The method uses 
an interviewer to instruct small groups of airmen in both the nature of 
the procedure and the value of such a study. The respondents then 
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describe incidents from memory. This retains the advantages of the 
individual interview in regard to personal contact, explanation, and 
availability of the interviewer to answer questions. 

Daniels (1976) developed a two page questionnair e and demonstrated 
its potential towards obtaining useful results in a safety school environ- 
ment. Appendix A contains completed samples of the report forms which 
are representative of the anecdotes elicited during this study. The 
initial presentation which was read verbatim to each group prior to 
report completion is reprinted in Appendix B. Forty-five minutes was 
alloted for each evolution. 

The following guides to development were adhered to in order to 
develop adequate means of collection and recording: 

1. The introductory remarks were standardized in order to 
maintain continuity across safety school classes. 

2. The report form stresses anonymity in order to avoid 
inadequate responses due to fear of reprisal, evaluation, 
or social pressures. 

3. The initial remarks were tailored to create a general 
atmosphere of cooperation with the respondents. 

4. The recording technique was engineered to be simple and 
thus easily understood by all participants; it also required 
a minimum of time and skill. 

5. Provision was made for eliciting the necessary control 
information - aircraft model, crew position, etc. 

6. The record form was amenable to easy handling and 

sorting. ^ 

7. The evaluator was not searching for particular problems; 
thus there was no attempt to direct questions towards any 
specific area. 
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C. ANALYSIS OF DATA 



The purpose of the data analysis stage is to summarize and des- 
cribe the reported incidents in an efficient manner so that they may be 
applied to problems of design. The aim is to increase the utility of the 
information while sacrificing as little as possible of its comprehensive- 
ness, specificity, and validity. 

The raw material for the analysis consisted of a descriptive report 
for each incident elicited. It should be recognized that each report is a 
unique anecdote describing a unique event. Thus each incident reported 
was transcribed, along with all control data, onto three by five cards 
and considered as one data point. 

The procedures used at this stage were proposed by Flanagan (1962) 
the general steps involved in the process are reproduced in Figure 4. 
There are three primary areas that are of critical importance which 
will be discussed in detail. However it is worthwhile to keep in mind 
some observations taken from a study of accidents by Thorndike (1951): 

''Note that the issue with respect to a particular way of 
analyzing accidents is not whether it is right or wrong. 

Any way of analyzing and any set of categories is 'right' 
as long as the categories are non-overlapping and are 
collectively inclusive. The fundamental issue is whether 
the system of analysis is the most useful one, that is, 
whether it can best serve as the basis for analyses which 
will answer practical questions and suggest courses of action 
which are likely to change things for the better. " 

a. Frame of reference 



Any system orientation will be based upon the judgment of the 
individuals performing the analysis; but, in general, the primary 
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Figure 4. 



GUIDE FOR ANALYSIS OF REPORTED DESIGN DISCREPANCIES 



1. Select general frame of reference. 

2. Sort a sample of incidents into a few piles in accordance with the 
frame of reference selected. 

3. Formulate tentative headings for major areas. 

4. Sort additional incidents into these major areas; set up new 
sub-categories as they appear necessary. During this process 
time will be saved if all incidents which are so similar that 
they will remain together regardless of changes in category 
definitions are clipped together and treated as one unit. 

5. After a substantial portion of the incidents have been classified, 
prepare tentative definitions for major headings and generalized 
statements for each of the main categories of requirements. 

6. Make a tentative selection of the level of specificity- generality 
to be used in reporting the definition. 

7. Redefine major areas and categories as necessary while incidents 
are being classified. 

8. After all incidents have been classified, review definitions and 
revise where necessary. 

9. Record the classification of each incident on the back of the card. 

10. Have an independent check made on the classification of all 
incidents . 
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consideration should be the uses to be- made of the data. It was deter- 
mined that the primary thrust of this study was to be an exploration 
into the problems exhibited across aircraft types and communities. 

The purpose is to demonstrate that common problems exist and that 
they can be analyzed. It was desired to determine a design deficiency 
stratification that would encompass each and every data point available; 
hypotheses could then be generated and provide inspiration for recom- 
mending further work. 

b. Category formulation 

The general process by which the data was evaluated and 
stratified involves category formulation. This portion of the procedure 
is more subjective than objective, but, in general, the process involves 
grouping similar incidents under major headings, then into categories, 
deriving subgroups, and finally formulating descriptive statements 
where appropriate. 

The initial step dealt with the derivation of a trial classifica- 
tion system. Reports published on studies similar to the current effort 
(see history) plus the author's experience as a fleet aviator, provided 
the initial insight, experience and judgment for filing a small sample of 
incidents into areas consistent with the frame of reference. These data 
aggregations then became the initial major headings and were loosely 
defined. Additional incidents were then classified under major headings 
into subheadings called categories. This step was iterative with regard 
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to the trial and revision of these categories; in fact, the classification 
system as a whole was subjected to a constant evolution throughout the 
entire process. The goal sought was a workable classification system 
which encompassed all incidents and had a non- overlapping and distinctly 
defined format. 

A need for subgroups within the categories soon became 
apparent. These were developed on an entirely inductive basis; they 
consisted of groups of incidents describing very nearly the same design 
discrepancies . 

Each incident was carefully read and re-read before assigning 
it to the system. If an incident could not be readily assigned, a revision 
in the stratification scheme was made by broadening existing headings, 
categories, and/or subgroups, or by adding new ones as appropriate, 
c. Level of specificity/generality 

This area concerned the specificity achieved in particular 
incidents versus the simplicity of a relatively smaller but more useful 
number of headings. Several considerations were adhered to at this 
level: 

1. A clear cut and logical organization was sought which 
could be presented in standard outline form. 

2. Titles were sought for all headings, categories, and 
subgroups which could convey meaning in themselves. 
Nonetheless, definitions and descriptive statements 
were still provided. 
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3. The stratification scheme chosen was that deemed 
maximally useful in highlighting design problem 
areas, yet which was still significant at each level 
in terms of frequency of occurrence. 

4. A descriptive statement was written for each group 
of duplicate incidents or for each single incident 
representing a distinctly different design discrepancy. 

5. Descriptive statements were written in terms suf- 
ficiently general to cover basic similarities yet 
specific enough to delineate specific problem areas. 

After the initial system evolution it became obvious that the 
number of categories involved resulted in an unwieldy system. A 
reduction was made by combining similar subgroups and writing new 
descriptive statements. The essence of the subgroups was preserved 
but some specific detail (common to some but not all of the incidents) 
was lost. 



After all classification was completed and the level of speci- 
ficity/generality deemed satisfactory, the definitions and descriptive 
statements were closely re-examined in terms of the actual incidents 
as now classified. Alterations were made where required. 
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VI. RESULTS 



Results were elicited from five separate safety school classes 
during the period October, 1975, to February, 1976. A total of 137 
aviators were questioned resulting in 286 reported design discrepancies. 
The procedure yielded an average of 2. 09 incidents per respondent. 

The results suggest that fleet aviators have something to say about 
design deficiencies. Fleet experience was indicated by averages of 
1816 total flight hours per respondent, and also 201 flight hours /respond- 
ent in the aircraft reported on during the past six months. 

The sample as a whole contained data points covering twenty-six 
different Navy aircraft, an indication of the breadth of the information 
across communities. Table 1 presents the relevant control data in 
tabular form broken down by individual aircraft and respondent status 
(Pilot, NFO). The table reveals some notable influences which occur 
when using safety school data as a microcosm for the fleet. Pilots 
significantly outnumber NFO's by a 8 to 1 margin, a significantly greater 
ratio than might be expected. Only the F-4 community seems to have 
adequate NFO representation. It was also noted that 49% of the data 
was received from the P-3, F-4, A-6, and A-4 communities. In fact, 

72. 8% came from the fighter, attack, and ASW communities. However, 
these are also clearly the main components of the present day force 
structure. 
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TABLE I SUBJECT DATA - continued 
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122 15 137 247 39 286 100 



Table II presents the general headings and categories developed 
through application of the Problem Incident Reporting Technique. 
Statistics included are total discrepancies per heading, total per 
category, and frequency of report per heading. The complete analysis 
is presented in Table III. It consists of a systematically collected set 
of 31 subgroups, 10 categories, and 5 headings elucidating those design 
problems viewed as significant by fleet operators. Each subgroup is 
quantified by number of discrepancies and frequency of report. Descrip- 
tive statements and definitions are provided where appropriate. 

In terms of frequency or relative proportions, the grouping under 
the second general heading. Aircraft Cockpit Layout Errors, seems 
most important. One hundred and forty- seven documented incidents, 
representing 51. 4% of all reported design discrepancies, fall under 
this heading. An examination of the subgroups under the three categories 
which evolved (control/display location, control/display geometry, and 
workspace area) reveals problem areas which might not be anticipated 
by an engineer lacking aviation experience and working with a stationary 
mockup. Twenty-five location errors which prohibit an effective visual 
instrument scan were reported. Eighteen discrepancies revealed that 
certain controls or displays were difficult for aviators to use not because 
of design, but because they were hidden behind other aircraft fixtures. 
There were 28 reports that aviators could not reach or activate a con- 
trol when properly seated in the aircraft. In addition 53 other discrep- 
ancies were reported in the workspace area, an aviator^s ' office 
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TABLE II 



BREAKDOWN OF INCIDENTS BY 
MAJOR HEADING AND CATEGORY 



CATEGORY HEADING 
INCIDENTS TOTAL % 



I. 


EQUIPMENT DESIGN 
A. Control 


39 


69 


24. 13 




B. Display 


30 






II. 


AIRCRAFT COCKPIT LAYOUT ERRORS 
A. Control and Display Location 


60 


147 


51.4 




B. Control and Display Geometry 


34 








C. Workspace Area 


53 






III. 


VISION 
A. Internal 


15 


23 


8. 04 




B. External 


8 






IV. 


ENVIRONMENT 
A. Environmental Control 


13 


27 


9. 44 




B. Life Support 


14 






V. 


SAFETY 

A. Inadequate Safety Factors 


20 


20 


6. 99 




TOTALS 


286 


T86 


100 
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TABLE III 



ANALYSIS OF 286 OPERATOR GENERATED 
NAVAL AIRCRAFT DESIGN DISCREPANCIES 



NO. 

OF 

INCIDENTS % 

I. EQUIPMENT DESIGN 



A. Control 

1. Distinction Errors: 15 

Errors resulting from inability to 
distinguish among switches in close 
proximity and/or confusion among 
switches similar in size and shape. 

2. Adjustment Errors: 6 

Operating control requires excessive 
visual attention because of "lack of 

feel" or because of excessive physical 
effort in actuating. 

3. Excessive Positive Actuation: 4 

Control requires a significant time 
period of actuation for operation. 

4. Reversal Errors: 3 

Control must be moved in a direction 
opposite to that necessary to produce 
desired result. 

5. Unintentional Activation: 11 

Control positioned/designed such 

that danger exists of inadvertently 
operating it without an awareness 
of doing so. 

B. Display 

1. Parallex: 11 

Difficulty in reading instrument as 
required because of angle from 

which it is viewed. 

2. Legibility: 6 

Errors which result from difficulty 

in seeing the desired information 
distinctly enough to interpret indica- 
tion properly. 



5.2 



2 . 1 



1. 4 

1.0 



3. 8 



3. 8 



2 . 1 
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TABLE III - continued 



NO. 

OF 

INCIDENTS 



% 



3. Interpretation; 4 

Instrument indication subject to 
misinterpretation with the result that 
subsequent actions may aggravate vice 
correct undesirable condition 

4. Unreliability From Poor Construction: 9 

Display does not attract attention and/ 

or not utilized due to poor design. 

II. AIRCRAFT COCKPIT LAYOUT ERRORS 

A. Control/Display Location 

1. Vertigo Inducing Location; 17 

Instrument requires visual check 

causing a distraction which, under 
certain circumstances, may result 
in a contradiction between the infor- 
mation received by a pilot through his 
vestibular (or labyrinthine) sense and 
through his visual sense. 

2. Location Prohibits Effective Scan: 25 

Control or display placed such that 

an effective visual instrument scan 
must be broken to activate/check. 

3. Controls, Displays Hidden Behind Other 18 
Aircraft Fixtures 

B. Control, Display Geometry; 

Problems associated with dimensional 

arrangement of crew station. 

1. Inadequate Placement, Displays: 6 

Dimensional layout of cockpit makes 
display difficult to see. 

2. Inadequate Placement, Controls; 28 

Dimensional layout of cockpit makes 
control difficult to reach/activate when 
pilot properly seated and harnessed. 



1. 4 



3.2 



6 



8 . 8 



6.3 



2 . 1 

9. 8 
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c. 



TABLE III - continued 

NO. 

OF 

INCIDENTS 

Workspace Area 

1. Operation and Maintenance Consoles. 

a. Insufficient Instrumentation: 17 

Operator does not have neces- 
sary instrumentation for proper 
performance of his duties. 

b. Lack of Standardization: 5 

Cockpit instrumentation not 
standardized across variations 

of the same aircraft type. 

2. Inadequate Storage: 3 

More stowage space required for 
publications and emergency equipment- 
items which could become missiles in a 
ditching /heavy turbulence situation. 

3. Lack of Well Lit, Well Situated 11 

Approach Plate Holder. 

4. Ingress/Egress: 13 

Problems associated with the crew- 
man's ability to exit or enter the crew 
station both normally and during an 
emergency. 

5. Comfort /Mobility: 4 

Problems associated with crewman's 
comfort and his ability to move around 

in the crew station in order to perform 
his duties. 



III. VISION 

A. Internal Vision: 15 

Problems which interfere with a crewman's 
ability to acquire internal cues (i. e. , light- 
ing too dim, too difficult to see instrument 
indication in sunlight). 

B. External Vision: 8 

Problems which interfere with crewman's 
ability to acquire external cues (i. e. , night 
display lighting reflects off canopy). 



% 

6 

1. 75 
1.0 

3.9 

4.6 

1.4 



5.3 
2 . 8 
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TABLE III - continued 



NO. 

OF 

INCIDENTS 



IV. ENVIRONMENT 



A. Environmental Control: 

Problems associated with environmental 
factors which have a direct behavior on 
operator performance. 

1. Temperature/ Ventilation 5 

2. Nois e/Vibration 8 

B. Life Support: 

Problems associated with an aviator's 
personal equipment, survival equipment, 



etc. 

1. 


Oxygen/Smoke mask 


5 


2. 


Protective clothing 


3 


3. 


Restraint gear 


5 


4. 


Relief tube 


1 



V. SAFETY 



A. Inadequate Safety Factors: 

Operator identified discrepancies which 
could potentially evolve into conditions 
not conducive to good safety practice. 

1. Air canopy problems 7 

2. Training/NATOPS procedures, lack of 3 

3. Ejection seat/seat pan 4 

4. Inadequate/insufficient safety features 6 

(i. e. , equipment hard to reach in an 
emergency, lack of weather radar, 

lack of safety checks). 



% 



1. 75 

2 . 8 



1. 75 
1 . 0 
1. 75 
. 035 



2.5 
1 . 1 
1.4 
2 . 1 
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all problems that could have been avoided by more knowledgeable 
direction in the design phase. Clearly, design engineers have problems 
in conceptualizing the plight of an aviator in an actual operational 
environment. 

Judging from frequencies the design of the control or display itself 
is next in importance. Sixty-nine such incidents were analyzed in 
Table III. Particularly noteworthy was the similarity between sub- 
groups which evolved from this report and those which Fitts and Jones 
(1947a, 1947b) constructed from pilot error analyses of aircraft con- 
trol operation and instrument interpretation. The twenty-nine year time 
span between the reports indicates the necessity for the transmittal of 
this type of information to designers. It also suggests the absence of 
corporate memory in Naval Aviation. ^ 

The discrepancies defined under the remaining three major headings 
comprised only 24. 47% of the total incidents. Yet it must be remembered 
that only one incident, if well documented, is of significant importance. 

Any factors which interrupt an aviator's visual perspective, either 
internal or external to his craft, can have a degrading effect on total 
system performance. There were 15 reported cases of excessively 
dim lighting at night, instruments that were unreadable in sunlight, and 
other problems which interfered with an aviator's ability to acquire 
internal cues. Nine incidents (such as night canopy relfection from 
display lighting) resulted from external vision interrupts. Here again 
are discrepancies that would be difficult to forecast from a mockup or 
simulator. 
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Also difficult to predict, but within the realm of human factors, are 
the environmental control and life support design problems elicited. 
Problems involving temperature, ventilation, noise, vibration, and life 
support equipment all affect a crewman’s perception of his environment 
and influence his performance. 

The final heading was a loosely defined grouping termed Safety. 
These were, in fact, discrepancies which operators felt could result in 
conditions not conducive to good safety practices. Some, such as lack 
of training/NATOPS procedures, might be viewed as foreign to the 
design process. Yet it is at this early stage where initial iterations 
for such procedures might be formulated. 

Table IV presents a breakdown of the classification system versus 
the four aircraft with the largest sample sizes - P-3, A-4, F-4, A-6 - 
measured in terms of frequency of responses per given aircraft. The 
table gives an idea of the dispersion of individual aircraft responses 
among the various headings and categories. These models have all 
undergone various modifications; the A-4 and F-4 are in the twilight of 
their careers. It is clear that the classification scheme represents 
design discrepancies which affect all individual aircraft and yet which 
have gone uncorrected by design engineers over a substantial period of 
aircraft evolution. If such problems have not been pinpointed in the past, 
there is little reason to expect them to be accounted for in new aircraft. 
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TABLE IV 



DISCREPANCY ANALYSIS VERSUS FOUR AIRCRAFT TYPES 









% 


% 


% 


% 








P-3 


A-4 


F-4 


A-6 


I. 


EQUIPMENT DESIGN 












A. 


Control 


2.5 


17. 6 


11.4 


6.2 




B. 


Display 


15 


3 


2.9 


18. 8 


II. 


A/C CONTROL LAYOUT ERRORS 












A. 


Control/Display Location 


10 


26.4 


17. 1 


25 




B. 


Control/ Display Geometry 


5 


20. 6 


17. 1 


15. 6 




C. 


Workspace Area 


40 


14. 6 


14. 3 


18. 8 


III. 


VISION 












A. 


Internal 


5 


3 


2. 9 


0 




B. 


External 


0 


3 


2.9 


12. 5 


IV. 


ENVIRONMENT 












A. 


Environmental Control 


7. 5 


3 


8. 6 


0 




B. 


Life Support 


12. 5 


5. 8 


5. 7 


3. 1 


V. 


SAFETY 












A. 


Inadequate Safety Factors 


2. 5 


3 


17. 1 


0 






100% 


100% 


100% 


100% 



41 



It can also now be said that two types of requirements arise when 
discussing design problems: 

1. Those inspired by specific aircraft deficiencies. 

Z. Those which are based upon a need by all aircraft. 

The current effort presents a stratification that can serve as the founda- 
tion for study of either requirement as the descriptive statements written 
for this analysis are generally applicable. A more specific and detailed 
analysis tailored towards either of the above directions would lead to 
more critical discrepancy areas for which specific remedial procedures 
might be devised. This relates directly to the question of, "How many 
incidents should be collected?" Flanagan (1962) suggests a criteria 
that would lead to a much larger sample than the present one; namely, 
adequate sampling is achieved when the addition of 100 incidents results 
in only two or three new discrepancies. This type of very detailed work 
could spell out specific trouble spots upon which attention could be 
focused for the purpose of understanding and diagnosis. 
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VII. CONCLUSIONS 



Although the level of specificity/ generality did not allow specific 
identification of error causing equipment, the results do provide a view 
of the problems which hinder operator performance. The derived design 
deficiency structure demonstrated that an identifiable body of common 
cause factors exists across a large number of different Naval aircraft. 
The constant nature of the problem suggests that the causes are rooted 
in the aviation system itself and that research should be directed towards 
conceptual deficiencies in the design of air weapons systems in general. 

Previous discussion established a link between equipment design as 
a system element and the operator in terms of an error causing overload 
conditions. Each of the design problems identified in this paper can 
induce such a state providing it occurs at a critical time during the 
mission. It is not the deficiency area per se that is important, but 
rather the interactive characteristics an area has with the system as a 
whole. As a result equipment may seem perfectly adequate to the human 
factors specialist when evaluated alone, while actually being inadequate 
when interacting with other system elements (including the operator) in 
the complex operational environment. 

Research needs to consider not only specific problems of equipment 
design, but to study a system in terms of the interaction of its elements. 



43 



System breakdowns , as well as individual equipment inadequacies, 
directly affect mission performance. Thus the Navy needs a system 
development program which includes consideration of total system 
operation (rather than merely operability) from the earliest of design 
stages. A start has been made in this direction by the approach presented 
in this study; deficiencies as deduced by the presented method reveal a 
unique view of problems as experienced in a systems context. 

With regard to the presented collection, classification, and analysis 
of the deficiency data, the author concluded that: 

1. Fleet aviators do feel that man-machine interface problems 
exist which hamper their mission performance. 

2. Aviators are able and willing to articulate and document 
these problems. 

3. The Problem Incident Reporting Technique contributes to 
an effective method for systematically collecting, analyzing, 
classifying, and describing the reported problem incidents. 

4. The continuation of the same discrepancy problem areas 
across generations of Naval aircraft and their continued 
inclusion in design suggests that operational problems do 
exist which cannot be anticipated by the design engineer 
using current techniques. 

5. The presented method can identify these operational problem 
areas which have been historically ignored. 
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6. The utilization of such information in the design stage can 

result in: 

a. improved performance in weapon systems cockpit and 
the man-machine interface design, 

b. improved manpower utilization and performance, 

c. fewer aircraft mishaps and an alleviation of mission 
degradation from equipment misuse, 

d. improved user acceptance, 

e. reduced training cost as aviators no longer must learn 
methods devised to circumvent design problems. 

7. Thus, a system has been devised for the total analysis of 

reported design discrepancies which yields: 

a. a new source of specific and derailed data which provides 
considerably more information on important design prob- 
lems than those currently available. 

b. a system for the classification and analysis of the data 
in such a way as to facilitate remedial action. 
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VIII. RECOMMENDATIONS 



This project represents the second iteration in developing a pro- 
gram for obtaining and utilizing design discrepancy data. The study 
used a questionnaire developed by Daniels (1976); however, the expe- 
rience accumulated in the course of this work indicates that valuable 
information was left undetected. Chapanis (1959) suggests that a pro- 
ject of this nature should elicit data that suggests cures. And, in fact, 
many aviators offered plausible solutions to the design problems which 
they described. In addition it would have been advisable to ask more 
specific questions in order to determine the phase of the mission which 
was affected, what the aviator did, and in what ways the problem de- 
graded the flight. In this way an idea of the criticality in terms of 
mission-performance could be devised. It is suggested that this be 
dealt with in a series of follow up projects; however, a solid starting 
point would be a record form developed by the American Institute for 
Research and used by Vasilas et al (1953) in a study of near accidents 
(see Appendix C). 

Eventually the study should be expanded to collecting design dis- 
crepancy data from all crew members on a fleet- wide basis. Practical 
procedures and channels must be devised for routing reports to squad- 
rons, for application within squadrons, and for routing back to the 
collection agencies. In addition channels for effecting the corrective 



46 



actions dictated by the analysis must be established. The facilities of 
the Naval Postgraduate School, the Naval Safety School, and the Naval 
Air Development Center should be explored with respect to establishing 
such procedures. 

It must be recognized that the data used in such a study as this is 
subjective. It is recommended that the design discrepancies which 
result from such data be substantiated with other observable data and 
documented reports. Aircraft inspections could be conducted and safety 
data screened for such validation purposes. The process when bolstered 
by additional information can only gain in credibility. 

Finally it is recommended that the resultant findings be forwarded 
to the human engineer with the recommendation that the solutions be 
included in the design specification. Meister and Farr (1966) in a study 
of the utilization of human factors by designers suggest: 

"The primary means of ensuring the inclusion of Human Factors 
in design is through the design specification .... We recom- 
mend that when consideration of Human Factors is desired, the 
customer insert strongly worded statements in the design 
specification " 

If an effective communications link is established from the squadron 
through the collecting agency to the contractor, perhaps the design 
problems that have plagued Naval aviators through generations can be 
mitigated. 
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APPENDIX A 



COCKPIT HUMAN FACTORS DEFICIENCY QUESTIONNAIRE 



SECTION I - RESPONDENT DATA 
^ 85 Date 19 Jan. 



\ Piioy 



NFO (check one) 



Total Flight time 2150 
Aircraft flown most in last 6 months. 



A-4M 

Type (ECP 1120) 



Hours 200 



Do you think the data we are trying to get is available elsewhere? Yes 
If yes, where? 



No 



SECTION II - IDENTIFICATION OR 


DESCRIPTION OF DEFICIENCY 
ITEM 1 ITEM 2 ITEM 3 


ITEM 4 


Type(s) of aircraft involved 


A-4M 


A-4M A-4M 


A-4M 


Name of deficient item 


ARC-159 

(placement 


nosewheel fuel 
steering trans 

_.:^tton _ Trakt 


AOA 

ind. 


Component ID (if known) 




(dual “ 

function) 




Manufacturer (if known) 




Douglas 


Douglas 


Was this item/situation standard? Yes No 
i. e. Did it occur on all aircraft in type? yes 
If NO, describe below 


yes yes 


yes 



SECTION III - BRIEFLY DESCRIBE THE SITUATION AND ITS POTENTIAL 

EFFECTS 

Item 1 

■'In the new A-4M with ECP 1120 (HUD), the cockpit lay-out has improved 
but, there are still some environmental problems. The ARC 15 9 a new UHF 
radio is the best I have ever had the opportunity to use. But, it is still 
located down on the console; it was moved from the right console to the left 
which was an improvement - facilitating keeping right hand on stick. How- 
ever, it is just about abeam one's left upper thigh. When making a channel 
change especially night, IFR, formation it is necessary to look down and to 
the left to read the new frequency. This is dangerous and impractical. 

Mount the radio on the left upper instrument panel or have a readout from 
the radio displayed in that position. The OV-10 is a good example of radio 
placement. It makes it much more comfortable and less vertigo inducing 
to switch frequencies while maintaining good look-out doctrine. " 
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APPENDIX A - continued 



Item 2. 

"The nose wheel steering button has a dual function in A/C with the 
(HUD). During weapons delivery if the master armament switch is 
cycled on-off, as in coming off a run, the pipper and bomb actuation 
line disappears. In order to regain this presentation the pilot must 
depress the nosewheel steering button. No one knows why this feature 
was even incorporated to begin with. The presentation does not impair 
one's view and as a result of depressing the nosewheel steering button 
many pilots have already inadvertently depressed the bomb pickle button 
dropping ordnance anywhere and everywhere. I suggest removing this 
feature of the nosewheel steering (dual purpose) and merely leaving the 
pipper and bomb actuation line always displayed. All pilots in my present 
squadron concur - it's a ridiculous feature. " 



Item 3. 

"There is no longer a fuel transfer light in the new A-4M displayed on 
the enunciator panel. The NATOPS makes reference to it still being 
there but it is not. A change has already been submitted to the NATOPS 
manual. However, the pilots would like to have the light back. There 
is presently no way to discriminate between a stuck float valve and a 
fuel transfer failure. They are both indicated by a drop in fuel quant, 
to approx. 1100 lbs. followed by a low fuel light. " 



Item 4. 

"The AOA indicator is in a poor position for the pilot to see when seat 
is in upper position for landing. Raise the whole indicator for better 
landing A/S monitoring. " 
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il l II 



APPENDIX A _ continued 



COCKPIT HUMAN FACTORS DEFICIENCY QUESTIONNAIRE 
SECTION I - RESPONDENT DATA 



#49 



Date 




NFO 



Total Flight time 



10 Nov 75 
(check one) 
4500 



Aircraft flown most in last 6 months. Type P-3C Hours 30 

Do you think the data we are trying to get is available elsewhere? 
Yes No . If yes, where? 





SECTION II - IDENTIFICATION OR DESCRIPTION 






OF DEFICIENCY 


ITEM 1 


ITEM 2 


ITEM 3 


Type(s) of aircraft involved 


P-3C 


P-3C 


P-3C 


Name of deficient item 


Approacn 

plate 

holder 


tiignt 

director 

indicator 


vertical 

speed 

indicator 


Component ID (if known) 


Manufacturer (if known) 


Was this item/ situation standard? 








Yes No 

i. e. Did it occur on all aircraft in type? 
If NO, describe below 


yes 


yes 


yes 



SECTION III - BRIEFLY DESCRIBE THE SITUATION AND ITS POTENTIAL 
EFFECTS 
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APPENDIX A - continued 



Item 1 

"In the P3C aircraft there is no holder provided for the approach 
plate/departure plate book. Many 'made do' adaptations have been 
designed and used by various individuals. These jury rigs usually 
involve clipping a sheet aluminum plate somewhere under the glare- 
shield or at the edge. Most of these locations cause certain instrument/ 
indicators to be hidden from the pilots scan. Also none of these locally 
manufactured solutions are lighted causing night approaches to be less 
safe by the pilot /co-pilots moving the approach plate to a light source. " 



Item 2 

"The Flight Director Indicator (FDI) in the P3C provides the primary 
horizon reference under IFR conditions. The 'pipper' is non-adjustable 
by the pilot. Depending on the height -of-eye of the viewer it will in- 
dicate a climb or descent when in fact the aircraft is straight and level. 
It causes instrument fixation and disorientation during the instrument 
flight. Without an operational AFCS it is extremely difficult to fly 
straight and level. " 



Item 3 

"The pilots VSI is too small for its position on the instrument panel, 
It should either be the larger type gauge or be repositioned. (Note; 
They might have taken the c/p VSI out of the P3C. I'm not sure as 
I haven't had a chance to fly in the rt. seat. )" 
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APPENDIX B 



INSTRUCTIONS FOR FILLING OUT THE COCKPIT HUMAN FACTORS 
DEFICIENCY QUESTIONNAIRE 

There is presently no large data base, from the pilot's point of 
view, for problems that the pilot faces caused by the cockpit and its 
equipment. It may be too late or expensive to change the specific items 
or address specific problems that relate to known deficiencies today. 
However, the hope is that a data base made up of such problems, 
statistically relevant and properly presented in the DESIGN stage of 
tomorrow's cockpits may save us from having these same or similar 
problems on our next tour. 

The number and date are for statistical purposes only and have 
been filled out by the instructor. Since the questionnaires have been 
randomly distributed, you may anonymously speak your mind. 

Please operate independently, since if all the P-3 drivers come 
up with the same item it will tell us one thing (which we don't want). 

If you all come up with a different item, that will tell us something 
also. 

It is anticipated that 95% of the items will be UNCLASS, however, 
if the item you wish to relate is classified, please so mark your 
questionnaire and personally let me know so that the special markings 
and handling may be complied with. 
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I 




Now specifically, think back on your last flight experience and 
remember some cockpit piece of equipment, murphy, or design 
feature that was a problem (couldn’t reach, see, feel, hear, actuate, 
etc. ) Almost any specific item that caused some difficulty, lack of 
coordination, hazard, noise, temperature, etc. will be helpful. If 
you have two or three, so much the better. Space is provided for 
three items. 

Everybody got an item? If not, keep thinking back on flights 
until you get at least one. OK, carry on! 
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APPENDIX C 



HAZARDOUS INCIDENT REPORT 



(1) Base: Ac ft type: Date: 

check ( ) or complete the following applicable data: 

DAWN DAY DUSK NIGHT Clearance: VFR IFR LOCAL 

PHASE OF FLIGHT: 

TAXIING TAKEOFF CLIMB CRUISE DESCENT APPROACH 
LANDING OTHER TYPE OF MISSION: 



NARRATIVE DESCRIPTION OF INCIDENT 

(2) Give enough details so that the story will be clear. Use other side 
if more space is needed. 

A. Describe what led to the hazardous incident 



B. What did you do? 



C. What was the immediate result? 



D. What would be the best way of handling the situation if it happened 
a gain ? 



E. How could such a hazard be eliminated or avoided? 



(3) Check your crew position 

Pilot Co-pilot Bombadier Navigator Observer 

Radar operator Engineer Gunner Other 

Developed by 

The American Institute for Research 
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